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BRIEF  DESCRIPTION  OF  PORTFOLIO: 

Developing  mathematical  theory  and  algorithms  based  on  the 
interplay  of  dynamical  systems  and  control  theories  with  the  aim  of 
developing  innovative  synergistic  strategies  for  the  design,  analysis, 
and  control  of  AF  systems  operating  in  uncertain,  complex,  and 
adversarial  environments. 


LIST  SUB-AREAS  IN  PORTFOLIO: 


/ 


General  Control  Theory:  optimal  control,  adaptive  control,  stochastic 
control,  hybrid  control 

Distributed  Multi-Agent  Control:  path  planning,  decision  making, 
sensing,  task  allocation  with  adversarial  and  stochastic  elements  with 
incomplete  information  and  communication  constraints 

Emerging  Applications:  Quantum  Control,  Vision-based  Control 

V&V  of  embedded  Systems 

Mixed  Human-Machine  Interface 

Control  of  Distributed  Parameter  Systems 


DISTRIBUTION  A:  Approved  for  public  release;  distribution  is  unlimited. 


2 


Key  Technology  Areas  for 
Autonomous  Systems 


Autonomy  from  the  Dynamics  and  Control  Point 

Key  Technology  Areas:  (from  Tech  Horizons  Report  2010  "  United  States  Air  Force 


Chief  Scientist  (AF/ST)  --Report  on  Technology  Horizons:  A  Vision  for  Air  Force  Science  &  Technology  During  2010-203o) 


•  Autonomous  systems 

•  Autonomous  reasoning  and  learning 

•  Resilient  autonomy 

•  Collaborative/cooperative  control 

•  Autonomous  mission  planning 

•  Embedded  diagnostics 

•  Decision  support  tools 

•  Sensor-based  processing 

•  Human-machine  interfaces 
Grand  Challenges: 

•  Trusted  Highly-Autonomous  Decision-Making  Systems 

•  Fractionated,  Composable,  Survivable,  Autonomous  System* 
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•  Objectives:  To  address  fundamental  problems  in  distributed  control  of 
networked  systems  with  non-traditional  constraints  and  adversarial  action. 

•  Challenges:  A  comprehensive  research  program  that  deals,  within  a 
multi-agents  context, 

-  with  decentralization, 

-  localization  of  objectives, 

-  lossy  communication  and  information  exchange, 

-  incompleteness  of  information, 

-  adversarial  interference, 

-  multiplicity  of  objectives, 

-  coordination  through  multi-level  interaction. 

•  Modeling  Framework:  deterministic,  stochastic  and  hybrid  structures, 
cast  in  discrete-  as  well  as  continuous-time  settings,  with  a  strong 
mathematical  underpinning. 
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Key  Portfolio  Research 
Challenges 


•Autonomous  Dynamic  Mission  Planning  :  Hybrid  System  Formulation 
mix  of  finite,  discrete  state  (decision  variables)  and  continuous  Dynamics, 
game  theory 

•  Human-Machine  Interactions  and  Operations:  dynamics  queuing  theory 
for  task  allocation  for  human  operators,  formal  models  of  decision  making 

•  Incorporation  of  Uncertainty  in  Mission  Environment,  Model  Parameters: 
Adaptive  Control  (LI -Control  theory),  Stochastic  Control 

•  Adversarial  Behavior:  Game  theoretic  Approaches,  Stochastic  game 
theory  (mean-field  theory) 

•  Incomplete  Information:  incorporation  of  learning  into  games,  task 
allocation,  planning 

•  Computational  Issues:  Computational  Nonlinear  Control  Theory 

•  Emergent  Applications:  Quantum  Control,  Vision  Control 
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Hybrid  Control  for  Multi-Agent  Systems  in  Complex 

Sensing  Environments 

(Geir  E.  Dullerud,  UIUC) 


Dynamical  model: 

X  =  fot(x,u) 
z  =  h0t(x,u) 

Gt  state  of  a  specified  automaton. 

At  time  t  the  values  z(t)  and 

6>(r),  for  preview  horizon  r  E  [t,t+H] 

are  available  for  decision  making. 


Example: 

•  preview  of  planned  trajectory  available  out  to  a  finite  known  future 
(shown  in  red). 

•  governing  vehicle  parameters  known  from  present  to  H  time 
units  into  future. 

•  strategic  objectives  known  over  the  dynamic  interval  [t,t  +  H]. 


Captures  situations  where:  upcoming  environment  (e.g.,  obstacles)  can  be  previewed,  for 
instance  via  vision-sensing  or  radar;  also  scenarios  where  mission  objectives  are  fixed 
over  a  window  but  evolving. 

A  hybrid  system  class  with  special  structure;  this  structure  can  be  exploited. 

Result:  Provide  a  complete  convex  solution  (analysis  and  synthesis)  to  discrete-time  linear  model  case,  for  both 
deterministic  and  stochastic  performance  metrics  (next  chart). 

Form  of  LTI  solution  indicates  approach  for  nonlinear  case. 
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Formulations: 

Model: 

xt+l  =  Aetxt  +  Betwt 
zt  =  Cotxt  +  ^te¬ 
stability  criterion:  uniform  exponential. 
Performance  measures  treated: 

(a)  max  gain:  sup -7— 7-; 

w  ||w|| 

t+H 

( b )  windowed  variance:  ^  E|z(fc)l2  <7 et....,et 

k=t  ’  H 


Induced  graphs  and  swtiching  paths: 

Discrete  dynamics  governed  by  a  reg¬ 
ular  language,  realized  by  automa¬ 
ton.  More  general  languages  possi¬ 
ble. 


Control  policy  depends  on  an  induced 
graph,  based  on  both  future  and  past 
automaton  paths. 


Example  Automaton: 

1,2, 3,4  denote  state  values  0t: 
transition  lables  a,  / 


Example  Induced  Graph: 

(k)  denotes  current  state; 
for  case  L  =  1 ;  H  =  1 . 


Solution  characteristics 

-  provides  first  solution  to  40-year  old  open 
problem  (solution  obtained  is  actually  more 
general). 

-  Analysis  and  synthesis  problems  reduced  to 
convex  programming  (SDP). 

-  Can  design  discrete  logic:  yields  a  hybrid 
separation  principle. 

-  applies  to  Markovian  jump  systems:  almost  sure 
uniform  stability,  almost  sure  performance. 

Current:  nonlinear  version  via  dynamic  programming. 
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t  2010  MURI  (PI:  Tamer  Ba§ar,  UIUC) 
Multi-Layer  and  Multi-Resolution  Networks  of 
Interacting  Agents  in  Adversarial  Environments 


Vision  and  Overarching  Goal: 


To  address  fundamental  issues  that  arise  in  networked  heterogeneous 
agents,  among  which  are: 

complex  interactions,  uncertainty  and  adversarial  actions,  trust, 
learning,  humans-in-the-ioop,  information  and  communication,  and  design 
of  architectures  to  facilitate  generation  and  transmission  of  actionable 
information  for  performance  improvement  under  different  equilibrium 
solutions. 


MLMR  Games:  Games  played  at  different 
levels,  interacting  through  their  outcomes, 
action  spaces,  and  costs 
"Games  within  Games":  Zooming  in  and  out 
providing  game  structures  with  different  levels 
of  granularity 
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•  A  system  “controlled”  by  many  agents 

•  Each  agent  may  have  its  own  objective 

•  Question:  Does  an  equilibrium  or  optimal 
solution  exist? 

-  Assumptions:  The  game/team  parameters  (system, 
actions,  rewards/costs,  etc.)  are  known, 
computational  power  is  unlimited,  and  information 
is  not  faulty. 

•  What  if  some  or  all  of  these  assumptions  do 
not  hold? 
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Sample  Result : 

Heterogeneous  and  Hybrid  Learning 


Learning  algorithms  are  essential  for  applications  of  game  theory  in  an 
adversarial  environment. 

-  No  knowledge  of  your  own  payoff  function 

-  No  knowledge  of  the  payoff  function  of  your  adversary 

-  No  knowledge  of  the  action  spaces  of  the  adversary. 

Players  have  different  levels  of  rationality  and  intelligence 

-  Active  learner  vs.  passive  learner 

-  Fast  learner  vs.  slow  learner 

-  Homogeneous  learner  vs. 
heterogeneous  and  hybrid  learner 

Players  do  not  interact  all  the  time. 

A  two-person  security  game 


Attacker 


Internet 


Security  Policies: 

*  Device  Configurations 

*  Honey  net  Deployment 

*  Knowledge  Sharing 

*  Security  investment 

*  Routing  Protocol 

*  Privacy  Setting 


Firewall/IDS 


Router 


Administrator 


Workstation 


Server 


Result:  The  (Nonzero-sum  Stochastic  Game)  NZSG  with  unknown  states 
and  changing  modes  admits  a  state-independent  Nash  equilibrium . 
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GIDEANS:  Attack  and  Defense 
of  the  Network  (D.  Castanon,  BU) 


•  Games  where  distributed  attackers,  with  partial 
information,  attack  an  intelligent  network,  also 
with  limited  information 

-  Network  adapts  its  operations  to  attack  outcomes 

-  Network  also  may  have  partial  information,  distributed 
intelligence 

-  Attacks  may  be  “hard”  (attrition  of  components)  or  “soft” 
(information  corruption  or  denial) 

-  Interested  in  both  defense  and  attack  strategies 
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Uncertainty  and  Partial 
Information 


•  Prior  work  on  stochastic  Network  Interdiction  (Wood  et  al  ’98,  ‘02,  ...) 

•  Actions  of  attack  have  uncertain  outcomes 

•  Outcomes  perfectly  observed  by  defender 

•  Result:  Stochastic  minimax  problem  -  Stackelberg  game 

•  Major  Challenges 

-  Combinatorial  complexity  for  computing  strategies  even  in  perfect 
information  case 

-  Asymmetric  information  among  players  leads  to  difficulties  in 
decomposition  algorithms  such  as  dynamic  programming 

•  Approaches 

-  Approximate  game  analysis  and  solution  techniques  based  on 
performance  bounds 

•  Stochastic  or  approximate  dynamic  programming 

•  Minimax  performance  where  network  reacts  to  attacks 
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•  New  algorithms  for  solutions  of  single  stage 
stochastic  network  interdiction  problem 
approaches 

•  Exploits  new  performance  bounds  integrated  with  novel  branch  and 
bound  search 


•  3  orders  of  magnitude  faster  than  previous 

•  New  results:  algorithms  for  two- 
stage  attacks  for  general  networks 

•  Adaptive  information  exploitation  by 
attacker 

•  Extension  of  above  performance 
bounds  and  search  techniques  for 
fast  algorithms 


nn  nn  n  _ 
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t-Nash  Mean  Field  Games 

Peter  E.  Caines  (McGill  U.  Montreal) 


Overall  Objective:  Decentralized  decision-making  for  stochastic  dynamic  games. 

•  Large  Population  Stochastic  Dynamic  Games:  Large  ensemble  of  stochastic  dynamic 
agents  seeking  to  maximize  their  individual  and  collective  interest. 

•  Key  Intuition  from  Statistical  Mechanics:  Extremely  complex  large  population 
particle  systems  may  have  simple  continuum  limits  with  distinctive  bulk  properties. 

•  Nash  Equilibrium  Theory  for  Large  Population  Stochastic  Dynamic  Games  (4  elements): 
Individual  agent  systems  with  cost  functions  depending  on  the  states  of  the  population 

•  Element  I:  Individual  Agent  Dynamics 


dxi  -  f  O,. ,  ui  )dt  +  crdw . ,  1  <  i  <  N 


Element  II:  Individual  Costs  Depending  on  Mass  Behaviour 


T 


Vt  :=  Equilbrium  value 
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IVIass  influence 


Play  against  mass 


•oSo 

o§o 


Element  III:  oo-population  Nash  equilibrium  best  response  (BR)  controls 
by  Mean  Field  Game  (MFG)  equations 


u . 


given 


[um}-dtV{t,x)  =  MUBU[f{x,u)diy{t,x)  +  L{x,u,n)]+—  dlV(t,x\ 
[BR]  u°(t,x)  =  cp{t,x  |  ju{t,x)),  ju  =  generic  agent'  s  state  distributi  on 
[FPK  ]  dtju(t,x)  =  -ax[/(x,M0)/u(/,x)]+-^-5^.//(/,x). 


Element  IV:  Apply 


u 


in  the  finite  population  game  to  obtain  approximate  Nash 


Equilibrium  (as  N  goes  to  infinity  exact  NE  holds  among  all  the  agents). 


fib, 

111 


m 
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ic  £-Nash  MFG  theory  and  Recent 

Advances 


•  Solution  of  decentralized  decision-making  problems  with  many  (minor) 
competing  agents 

■  Key  Intuition:  Single  agent's  control  =  feedback  of  stochastic  local  (rough) 
state  +  feedback  of  deterministic  global  (smooth)  system  behaviour 

Recent  and  Current  Advances  in  c-NMFG  Theory  (2007-2011): 

(i)  localized  problems  in  space,  (ii)  stochastic  adaptive  c-NMFG  control, 

(iii)  leader-follower  systems,  (iv)  consensus  seeking  systems,  (v)  flocking, 

(vi)  cooperative  (social)  c-NMFG  and  egoist-altruist  theory,  (vii)  major- 
minor  agent  c-NMFG  theory  (Huang  2010),  (viii)  nonlinear  Markov  systems 
theory  (Kolokoltsov  2011). 

3-D  Trajectory 


Optimal  Mass  Transport  for  Signal  Analysis  and 
Control:  Tryphon  Georgiou  and  Allen  Tannenbaum 


Analysis,  Comparison,  Mixing  of 
Distributions/Masses 

natural  ways  to: 

-  establish  correspondence 

-  interpolate,  average 

-  image/spectra  registration 

-  intensity/volume  preserving 

0  0  0  0  0 

0  #  4)  ®  ® 
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Basic  insight 


•  Engineer's  problem  of  transporting  a  pile  of  soil 
to  an  excavation  site  with  the  least  amount  of 
work 


Optimal  Correspondence: 

-  tools  for  interpolation,  averaging 

-  metric/geometry  on  distributions 

Leonid  Katnorovich  received  the  Nobel  prize  in 
1975  for  his  work  on  optimal  transport  in 
connection  to  resource  allocation. 
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Recent  Transitions 


Very  high  resolution  methods  have  been  developed 

for  applications  in  targeting  systems  (pointing, 
detection  via  antenna  arrays) 

contact:  Dr.  Dan  Herrick  (AFRL/DESA) 


High  resolution  spatial  time-varying  spectra 
Interpolated  via  optimal  transport  geodesics 


time  time 


time  time  time 


time  time  time 
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Transformational  advances 


Optimal  transport  averaging  of  distributions  lead  to 
new  techniques  in:  estimation,  nonlinear  filtering 


y=x+x 


an  alternative  to  Unscented  Kalman  filter 


Samples 

Samples 

*  Sigma  points 

J 

I 

l  +  + 

*  Sigma  points 

Uncertainty  ellipse 

1 

1 

L  * 

Uncertainty  ellipse  of  UT 

- Gaussian  Particles 

/ 

J 

1 

- Gaussian  Particles 

i 

!< 

1 

J* 

Uncertainty  ellipse  of  GP 
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4  i  Robust  Dynamic  Vision  Methods  for  Persistent 
Surveillance  and  Enhanced  Vehicle  Autonomy 


Mario  Sznaier  (Northeastern  Univ) 

•  Go  find  something  interesting/unusual 

•  Report  only  when/what  I  need  to  know  ^Flexible  Autonomy 

•  Take  action  within  given  bounds 


Extracting  actionable  information  from  a  “data  deluge”: 


In  all  cases  relevant  events  comparatively  rare  and  encoded  in  less 
than  0(1 0~6)  of  the  data _ 

—  Data  as  manifestation  of  hidden  dynamic  structures 

—  Problem  can  be  reduced  to  identification  of  hybrid  models 
described  by  sparse  graphs. 

—  Obtain  tractable  problems  by  combining  dynamical 

systems,  optimization  and  semi-algebraic  geometry  tools. 
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Metro 


Recal  effort  m  Lawrence 


Lawrence  mayor's  toes 
Me  signal  ores  m  ouster 
etton 


Oly  arrests  100 


Wvtey  Bulge*  was  hts 


Boston  potce  move  in 
Greenway.  scores 


Watnart  grocery  stores 


Dons  BelacK.  85;  played 
ludge  on  ‘Law  &  Order' 


Paula  EtteCmck.  56. 
ieadng  voice  m  struggle 
tor  gay  chnl  rights 


Albert  Rose*-.,  at  tOI; 
was  oldest  iving  former 
US  governor 


Mattapan  seeks  change 


Northeastern  developing  high-tech  security 
devices 

School  opens  facility  for  antiterrorism  research 

By  Akllah  Johnson  LOBE  STAFF  OCTOBF.Rog.20u 

‘  ARTICLE  VIDEO 


Vermont  tourtsts  focus 
on  storm's  aftermath 


This  is  the  piaoe  where  the  sensitivity  of  a  hulking  MRI  machine  will  be  harnessed  in  a  contraption  the  size  of 
a  toothpick. 


Protesters  arrested  at 


Law  school  says  it  has 
recovered  stolen 
$173,000 


Fatal  Shooting  stuns 


Businesses  throw 
support  behind  Maine 
casino  question 


Boston  EMS  crew 
detvers  baby  git  on  1-93 


Man  fataly  slabbed,  two 
women  hurl 


Pauine  Austin; 
developed  weati 
after  WWII 


CMtord  Olson.  Canarian 


And  this  is  the  place  where  two-story  buildings  will  be  tested  to  see  what  blast  force  they  can  withstand. 


This  is  Northeastern  University's  George  J .  Kostas 
Research  Institute  for  Homeland  Security,  a  new  Si2 
million  building  on  the  school 's  Burlington  campus  that 
still  smells  of  fresh  paint.  Professors  and  students  stood  i 
labs  in  various  stages  of  completion,  detailing  the 
domestic  security  applications  of  research  thatwil!  occur 
in  a  restricted  and  classified  environment. 

"The  mission  of  this  facility  is  to  assist  the  government 
with  the  protection  of  our  freedom,"  said  Northeastern 
alumnus  George  J.  Kostas,  who  funded  the  project,  the 
largest  capital  donation  in  university  history.*  We  lived 
through  the  Great  Depression,  the  Second  World  War . . . 
and  now  we  see  the  country  facing  an  even  greater 
challenge.  The  challenge  is  terrorism." 

Kostas.  who  graduated  with  a  degree  in  chemical 
engineering  in  1943.  built  a  Houston-based  synthetic 
rubber  manufacturing  firm  called  Techno-Economic 
Services.  But  before  Kostas  became  a  businessman,  he  was 
a  government  scientist  doing  classified  research  to  figure 
out  how  to  replenish  the  country's  dwindling  rubber 
supplies  during  War  World  II. 

After  the  bombing  of  Pearl  Harbor,  communications  were 
severed  with  the  country's  natural  rubber  suppliers  in 
Southeast  Asia ,  he  said.  Uttle  was  known  a  bout  synthetic 
rubber  at  the  time,  so  the  federal  government 
commissioned  a  group  of  scientists  to  learn  more.  Kostas 
was  one  of  them. 


A  look  at  one  of  the  projects 

One  i4  the  project*  sou®  to  be  iHuSrnvay  at  Northeastern 
University**  George  J.  Kmtat  Research  Institute  for 


A  face  tracking  UAV 

http://www.youtube.com/watch?v=z1k4zSFDWnM 
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Success  Stories 


Finding  correlations  in  complex  scenarios 

The  “Curse  of  dimensionality”  is  a  major  roadblock  in  achieving  flexible  autonomy. 

This  project  takes  the  first  steps  towards  a  “Compressive  Information  Extraction”  paradigm 
Unmanned  vehicles 
Flight  and  perch  sensors 
Human  in  the  loop  data  overload  avoidance 

Hitherto  unexplored  connection  between  systems  identification,  information  extraction 
and  machine  learning 
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Quantum  Linear  Systems 

Theory  I  Matt  James.  Ian  Peterson 

•  Why  Quantum  Control?  Is  there  anything  new  with  Quantum 
Control? 

•  Emerging  quantum  technologies  driven  by 

-  miniaturization  (microelectronics,  nanotechnology) 

-  exploitation  of  quantum  resources  (quantum  information  and 
computing) 


Demand  new  concepts  and  tools  from  control  theory  at  the  quantum  level 

•  Classical  linear  systems  theory  has  a  history  going  back  some  50  years,  to  the 
birth  of  modern  control  theory  with  Kalman's  foundational  work  on  filtering  and 
LQG  optimal  control.  Gaussian  Distributions  play  a  fundamental  role  in  classical 
linear  systems  theory. 

•  The  Pis  have  shown  that  classical  finite  dimensional  linear  systems  cannot 
generate  entanglement  in  a  pair  of  quantum  linear  systems  initialized  in  a 
separable  Gaussian  state  fundamental  limitation  of  classical  linear 
systems  when  used  to  control  quantum  systems. 
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w  Achievements  so  Far  and 

Need  for  More  Research 

•  Need  for  new  paradigms  for  quantum  stochastics:  non- 
commutative  probability  models  and  a  non-commutative 
stochastic  calculus  that  are  not  generally  studied  in  the  classical 
setting 

•  Need  for  new  paradigms  and  techniques  for  feedback  control  of 

quantum  systems,  as  the  classical  theory  cannot  handle  the  uniquely 
quantum  phenomena  (e.g.  entanglement) 

•  This  new  area  in  the  portfolio  started  with  one  grant  (Australia),  and 
has  continued  with  a  YIP  project  and  a  new  one  in  2011 . 

•  2011  ARO  MURI  project.  Still  there  is  need  for  support  of  more  core 
projects,  since  the  area  has  the  potential  of  impacting  both  theory  and 
applications 

New  methods  for  using  direct  physical  couplings  in  optimal  coherent  feedback  system  design. 

Differential  evolution  optimization  methods  in  coherent  control  of  linear  quantum  systems. 

Decentralized  coherent  robust  H-infinity  quantum  control. 

Approximation  of  linear  quantum  system  models. 

Single  photon  processing  by  linear  quantum  systems. 
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Summary 


•  Dynamics  and  Control  is  the  key  enabler  in  the  science  of  autonomy.  There 
is  need  for  more  collaboration  with  other  disciplines:  cognitive  science, 
network  science,  communication,  information  science 

-  New  MURI  and  AFOSR  Basic  Research  Initiative  (BRI)  topics  in  V&V,  Human-Machine 
interaction,  Control  and  Synthetic  Biology 

•  Close  collaboration  and  sharing  of  information  with  other  agencies:  ONR, 
ARO  and  NSF 

•  In  addition  to  the  core  theme  of  distributed  control  of  multi-agents,  the 
program  has  solid  efforts  in  control  of  MAVs,  hypersonic  vehicles,  smart 
materials  and  biological  systems. 

•  Support  of  fundamental  research  in  Control  and  Dynamics  while  considering 
challenging  and  relevant  application  areas  is  what  distinguishes  this 
portfolio. 
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